{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6d742657-8e6a-439d-a65a-3c76e73c8810",
   "metadata": {},
   "source": [
    "# Logistic Regression with SPU"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ddc86c8",
   "metadata": {},
   "source": [
    ">The following codes are demos only. It's **NOT for production** due to system security concerns, please **DO NOT** use it directly in production."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff9c4ba5-a4a8-46ec-96e3-611ef5e71dc5",
   "metadata": {},
   "source": [
    "[SPU](https://spu.readthedocs.io/) is a domain specific compiler and runtime suite, which provides provable secure computation service. SPU compiler uses [XLA](https://www.tensorflow.org/xla) as its front-end IR, which supports diverse AI framework (like Tensorflow, JAX and PyTorch). SPU compiler translates XLA to an IR which could be interpreted by the SPU runtime. Currently SPU team highly recommends using [JAX](https://github.com/google/jax) as the frontend.\n",
    "\n",
    "## Learning Objectives:\n",
    "\n",
    "After doing this lab, you'll know how to:\n",
    "\n",
    "* How to write a Logistic Regression trainning program with JAX.\n",
    "* How to convert a JAX program to a SPU(MPC) program painlessly.\n",
    "\n",
    "In this lab, we select [Breast Cancer](https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(diagnostic)) as the dataset. We need to decide whether cancer is malignant or benign with 30 features. In the MPC program, two parties will train the model jointly and each party would provide half of features(15).\n",
    "\n",
    "While, first, let's just forget MPC settings and just write a Logistic Regression trainning program with JAX directly."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "629aad7f",
   "metadata": {},
   "source": [
    "## Train a model with JAX"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9774618-798e-421f-9be0-4fb954d7c710",
   "metadata": {},
   "source": [
    "### Load the Dataset\n",
    "\n",
    "\n",
    "We are going to split the whole dataset into train and test subsets after normalization with `breast_cancer`.\n",
    "* if `train` is `True`, returns train subsets. In order to simulate trainning with vertical dataset splitting, the `party_id` is provided.\n",
    "* else, returns test subsets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "364a380e-9cea-42e3-8ab5-06635df97478",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import Normalizer\n",
    "\n",
    "\n",
    "def breast_cancer(party_id=None, train: bool = True) -> (np.ndarray, np.ndarray):\n",
    "    scaler = Normalizer(norm='max')\n",
    "    x, y = load_breast_cancer(return_X_y=True)\n",
    "    x = scaler.fit_transform(x)\n",
    "    x_train, x_test, y_train, y_test = train_test_split(\n",
    "        x, y, test_size=0.2, random_state=42\n",
    "    )\n",
    "\n",
    "    if train:\n",
    "        if party_id:\n",
    "            if party_id == 1:\n",
    "                return x_train[:, 15:], _\n",
    "            else:\n",
    "                return x_train[:, :15], y_train\n",
    "        else:\n",
    "            return x_train, y_train\n",
    "    else:\n",
    "        return x_test, y_test\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb400b1f-f516-4348-8812-e05a2e8f5e17",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Define the Model\n",
    "\n",
    "First, let's define the loss function, which is a negative log-likelihood in our case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0ffb9d32-4150-41cb-a6e9-65cbccf6568a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import jax.numpy as jnp\n",
    "\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + jnp.exp(-x))\n",
    "\n",
    "\n",
    "# Outputs probability of a label being true.\n",
    "def predict(W, b, inputs):\n",
    "    return sigmoid(jnp.dot(inputs, W) + b)\n",
    "\n",
    "\n",
    "# Training loss is the negative log-likelihood of the training examples.\n",
    "def loss(W, b, inputs, targets):\n",
    "    preds = predict(W, b, inputs)\n",
    "    label_probs = preds * targets + (1 - preds) * (1 - targets)\n",
    "    return -jnp.mean(jnp.log(label_probs))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7eaaf2b4-6491-471a-85fb-d63236e4a564",
   "metadata": {},
   "source": [
    "Second, let's define a single train step with SGD optimizer. Just to remind you, x1 represents 15 features from one party while x2 represents the other 15 features from the other party."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "71e01788-804f-4378-a268-b84c3940320a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from jax import value_and_grad\n",
    "\n",
    "\n",
    "def train_step(W, b, x1, x2, y, learning_rate):\n",
    "    x = jnp.concatenate([x1, x2], axis=1)\n",
    "    loss_value, Wb_grad = value_and_grad(loss, (0, 1))(W, b, x, y)\n",
    "    W -= learning_rate * Wb_grad[0]\n",
    "    b -= learning_rate * Wb_grad[1]\n",
    "    return loss_value, W, b\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eac6baac-280c-44b4-b8a0-9a6eb3a76908",
   "metadata": {
    "tags": []
   },
   "source": [
    "\n",
    "Last, let's build everything together as a `fit` method which returns the model and losses of each epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f15b0c71-5761-41aa-b70d-9ba242f8a795",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def fit(W, b, x1, x2, y, epochs=1, learning_rate=1e-2):\n",
    "    losses = jnp.array([])\n",
    "    for _ in range(epochs):\n",
    "        l, W, b = train_step(W, b, x1, x2, y, learning_rate=learning_rate)\n",
    "        losses = jnp.append(losses, l)\n",
    "    return losses, W, b\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dd8abd5",
   "metadata": {},
   "source": [
    "### Validate the Model\n",
    "\n",
    "We could use the AUC to validate a binary classification model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "132fcee4",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "\n",
    "def validate_model(W, b, X_test, y_test):\n",
    "    y_pred = predict(W, b, X_test)\n",
    "    return roc_auc_score(y_test, y_pred)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a27ff403",
   "metadata": {},
   "source": [
    "If you are interested, we could also plot loss after each epoch of trainning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "61b1837c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def plot_losses(losses):\n",
    "    plt.plot(np.arange(len(losses)), losses)\n",
    "    plt.xlabel('epoch')\n",
    "    plt.ylabel('loss')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "581fac73",
   "metadata": {},
   "source": [
    "### Have a try!\n",
    "\n",
    "Let's put everything we have together and train a LR model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ad002d29",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "auc=0.9320340648542417\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmh0lEQVR4nO3dd3hVVbrH8e+bRpMiEEFBBQTEgIAYaYHEhgQLcAUFHLCLIihFndG5U51yLSNNEUHUGRRFRUUsVEtCh4CAgIAUaYogIlUI5b1/nMNMZAATzMlOcn6f58kjZ2Xtc959HpNf1l5nr2XujoiISG7FBF2AiIgULQoOERHJEwWHiIjkiYJDRETyRMEhIiJ5Ehd0AQWhcuXKXqNGjaDLEBEpUhYsWPCduyce2x4VwVGjRg2ysrKCLkNEpEgxs/XHa9elKhERyRMFh4iI5ImCQ0RE8kTBISIieaLgEBGRPFFwiIhInig4REQkTxQcJ/H+kq8Z/9lmtPS8iMh/KDhO4q0Fm+j3+iJ6v7qQ7/dmB12OiEihoOA4iVG3XMJv0usxbflWrhqUybTl3wZdkohI4BQcJxEbY/S69Dwm3JdCYtkS3Dk6i4feXMzu/QeDLk1EJDAKjlyoV7Uc7/ZOoc9ltXlr4SbSB09n1prvgi5LRCQQEQ0OM0s3s5VmttrMHj5BnxvNbLmZLTOzV3O0P25mS8NfXXK0v2Bmi81siZmNM7PTInkORyXExfBg2/N5q1dLSsTFcNPzc/nThGX8mH24IF5eRKTQiFhwmFksMAxoByQB3cws6Zg+dYBHgBR3rw/0C7dfAzQBGgPNgAfNrFz4sP7u3sjdGwIbgD6ROofjueic0/ng/tbc2rIG/5z1Fdc8PZ3PNuwoyBJERAIVyRFHU2C1u69192xgLNDhmD53AcPcfQeAu28NtycBme5+yN33AkuA9HCfXQBmZkApoMA/K1sqIZY/ta/PmDubsT/7MJ2Gz+Ifk1eSfehIQZciIlLgIhkc1YCNOR5vCrflVBeoa2YzzWyOmaWH2xcD6WZW2swqA5cBZx89yMxeArYA9YCnj/fiZtbTzLLMLGvbtm35c0bHSKldmUn9U7m+SXWe+WQ1HYfNZOWW3RF5LRGRwiLoyfE4oA5wKdANeN7MKrj7FOBDYBbwGjAb+PdkgrvfBpwFfAF04TjcfaS7J7t7cmLif21glW/KlYznHzc04vmbk9m6ez/XPT2D5zLWcPiIbhoUkeIpksGxmRyjBKB6uC2nTcAEdz/o7uuAVYSCBHf/m7s3dvc2gIW/92/ufpjQ5a9OEao/T9okVWFyv1Qur3cGj01cQZcRs/nqu71BlyUiku8iGRzzgTpmVtPMEoCuwIRj+ownNNogfEmqLrDWzGLNrFK4vSHQEJhiIbXD7Qa0B1ZE8BzypNJpJRjevQmDuzRm1be7aTdkOi/PWa8lS0SkWInYnuPufsjM+gCTgVjgRXdfZmaPAlnuPiH8vavMbDmhS1EPuft2MysJTA9lA7uA7uHniwH+Ff6ElRGaC+kVqXM4FWZGx4uq0axWRX49bgm/H7+UKcu28ETnhpxZvlTQ5YmI/GIWDX8NJycne1ZWVoG/rrvzytwN/P2DL4iPNf7coT4dG1cjHIgiIoWamS1w9+Rj24OeHC/WzIwezc9lYt/W1KlSlv6vL6bXKwvZvudA0KWJiJwyBUcBqFG5DG/c3YKH29Xj4xVbaTs4k6laMFFEiigFRwGJjTHuSQstmHhG2ZLcNTqLB99czC4tmCgiRYyCo4DVq1qO8b1TuO/y2ry9cBPpgzKZtVoLJopI0aHgCEBCXAwPXBVaMLFkfCw3jdKCiSJSdCg4AvRfCyYO1YKJIlL4KTgCdnTBxFfvbMaBQ0foNHwWT05eoQUTRaTQUnAUEi1rV2Ziv9Z0alKdYZ+socOwmXzxza6gyxIR+S8KjkKkXMl4ngwvmLht937aPzOD4Z9qwUQRKVwUHIVQm6QqTOmfRpukKjw+aQU3PDeLdVowUUQKCQVHIVWxTALDbmrCkK6NWb11D+2GZPLPmes4otGHiARMwVGImRkdGldjSv80mteqxJ/eW86vRs1l4/f7gi5NRKKYgqMIqFq+JC/degmPd7qQzzfvJH1wJq/O3aDl2kUkEAqOIsLM6HLJOUzun8pF55zOb9/5nJtfnMfXP/wYdGkiEmUUHEVMtQqlePmOpvylYwMWrN9B20GZvJm1UaMPESkwCo4i6Ohy7ZP6pnLBWeV4aNwS7vxXFlt37Q+6NBGJAgqOIuycSqUZe1dzfn9tEjNWf0ebQZm8u2izRh8iElEKjiIuJsa4o1VNJvZtTa3EMvQdu4heryzkO20WJSIRouAoJmolnsa4e1r+e7OoqwZlMvHzb4IuS0SKIQVHMXJ0s6j3729FtQql6DVmIfe/9hk79mYHXZqIFCMKjmKobpWyvH1vSx5oU5eJS7/hqsGZTNNWtSKSTxQcxVR8bAz3XVGHd3u3olKZBO4cncUDbyxm54/aqlZEfhkFRzGXdFY5JvRpxX2X12b8os20HZTJpyu3Bl2WiBRhCo4ocHSr2rd7teS0knHc+tJ8Hnl7CXsOHAq6NBEpghQcUaTR2RV4/75W3J1Wi9fnb6TtoExmrf4u6LJEpIhRcESZkvGxPNLuAt68pyUJcTHcNGouf3h3KfuyNfoQkdxRcESpi889nQ/vb81tKTUYPXs97YZMZ/5X3wddlogUAQqOKFYqIZY/XlefsT2bc8SdG0fM5q/vL2f/wcNBlyYihZiCQ2heqxKT+qbyq2bnMGrGOq4eOp3PNuwIuiwRKaQUHAJAmRJx/LXjhbxyRzP2Zx+m0/BZPD5pBQcOafQhIj+l4JCfaFWnMpP6p9L54uoM/3QN1z09g8837Qy6LBEpRBQc8l/KlYznic6NeOnWS9j540E6PjuTgVNXkX3oSNCliUghoOCQE7qs3hlM6ZdG+0ZnMfSjL+k4bCZffLMr6LJEJGAKDjmp8qXjGdSlMSN6XMzW3ftp/8wMhn70JQcPa/QhEq0UHJIrbetXZUr/NNIbnMnAqas0+hCJYhENDjNLN7OVZrbazB4+QZ8bzWy5mS0zs1dztD9uZkvDX11ytI8JP+dSM3vRzOIjeQ7yHxXLJPB0t4t4rnsTvt2l0YdItIpYcJhZLDAMaAckAd3MLOmYPnWAR4AUd68P9Au3XwM0ARoDzYAHzaxc+LAxQD3gQqAUcGekzkGOL73BmRp9iESxSI44mgKr3X2tu2cDY4EOx/S5Cxjm7jsA3P3oet9JQKa7H3L3vcASID3c50MPA+YB1SN4DnICGn2IRK9IBkc1YGOOx5vCbTnVBeqa2Uwzm2Nm6eH2xUC6mZU2s8rAZcDZOQ8MX6LqAUyKSPWSK+kNzmRq/zTaafQhEjWCnhyPA+oAlwLdgOfNrIK7TwE+BGYBrwGzgWNvYX6W0Khk+vGe2Mx6mlmWmWVt27YtUvULcHqZBIZ2u4jnul+s0YdIFIhkcGzmp6OE6uG2nDYBE9z9oLuvA1YRChLc/W/u3tjd2wAW/h4AZvZHIBEYcKIXd/eR7p7s7smJiYn5ckJycukNqmr0IRIFIhkc84E6ZlbTzBKArsCEY/qMJzTaIHxJqi6w1sxizaxSuL0h0BCYEn58J9AW6Obu+pO2kDne6GPINI0+RIqTiAWHux8C+gCTgS+AN9x9mZk9ambtw90mA9vNbDnwCfCQu28H4oHp4faRQPfw8wE8B1QBZpvZIjP7Q6TOQU5dztHHoGmr6PDMTJZ/rdGHSHFgoQ8nFW/JycmelZUVdBlRa9LSLfxu/Of8sO8g911eh3svO4/42KCn10Tk55jZAndPPrZdP70ScUdHH1dfqNGHSHGg4JACkXPuY+vuA5r7ECnCFBxSoEKjj1SuaajRh0hRpeCQAnd6mQSGdL0ovOKuRh8iRY2CQwLTtr5GHyJFkYJDAqXRh0jRo+CQQkGjD5GiQ8EhhcbxRh+Dp63S6EOkkFFwSKGTc/QxeNqXGn2IFDIKDimUNPoQKbwUHFKoafQhUvgoOKTQO97oY9DUVWQf0uhDJAgKDikyjo4+rm14JkM++pL2z8zg8007gy5LJOooOKRIOb1MAoO7XsSom5PZsS+bjs/O5PFJK9h/8NgNIkUkUhQcUiRdmVSFKf3T6NykOsM/XcM1Q6ezYP2OoMsSiQoKDimyypeK5/HODRl9e1P2HzxC5+dm8Zf3l/NjtkYfIpGk4JAiL7VuIpP7p9K92bm8MGMd6UMymbN2e9BliRRbCg4pFk4rEcdfOjbgtbua4w5dR87h9+OXsufAoZ8/WETyRMEhxUqL8yoxqV9rbk+pyStz19N2UCbTv9wWdFkixYqCQ4qd0glx/OG6JMbd04IS8TH0eGEevxm3hJ0/Hgy6NJFiQcEhxdbF51bkw/tb0+vS83hzwUbaDsrk4xXfBl2WSJGn4JBirWR8LL9Jr8f43imULxXP7f/Mov/ri9ixNzvo0kSKLAWHRIWG1Svw3n2tuP+KOry3+GvaDMpk0tJvgi5LpEhScEjUSIiLYUCbukzo04oq5UpwzysL6T1mId/tORB0aSJFioJDok7SWeUY3zuFh9qez9Tl39JmYAbvLtqMuwddmkiRoOCQqBQfG0Pvy2rzwf2tOLdSGfqOXUTPlxfw7a79QZcmUugpOCSq1alSlrd6teR311xA5qpttBmYwZtZGzX6EDkJBYdEvdgY487WtZjUL5V6Vcvx0Lgl3PrSfDb/8GPQpYkUSgoOkbCalcswtmdz/ty+PvO/+p62gzIZM3c9R45o9CGSk4JDJIeYGOOWljWY3C+VRmeX53/fWcqvRs1lw/Z9QZcmUmgoOESO4+yKpXnljmY8dv2FLN28k7aDM3lp5jqNPkRQcIickJnRtek5TO6fSrNaFfnze8u5ccRs1m7bE3RpIoFScIj8jLMqlOKlWy/hqRsaserb3bQbMp0RGWs4dPhI0KWJBELBIZILZkani6szbUAaaXUT+b+JK+g0fBYrt+wOujSRAqfgEMmDM8qVZESPi3m620Vs2vEj1z49nUFTV5F9SKMPiR4KDpE8MjOua3QWUwekcc2FZzLkoy+59unpLNr4Q9CliRSIXAWHmfU1s3IW8oKZLTSzq3JxXLqZrTSz1Wb28An63Ghmy81smZm9mqP9cTNbGv7qkqO9T/j53Mwq56Z+kUioWCaBwV0v4sVbk9m9/xDXPzuTv76/nH3Z2q5Wirfcjjhud/ddwFXA6UAP4LGTHWBmscAwoB2QBHQzs6Rj+tQBHgFS3L0+0C/cfg3QBGgMNAMeNLNy4cNmAlcC63NZu0hEXV6vClP6p3JTs3MYNWMd6YOnM2v1d0GXJRIxuQ0OC//3auBld1+Wo+1EmgKr3X2tu2cDY4EOx/S5Cxjm7jsA3H1ruD0JyHT3Q+6+F1gCpIf7fObuX+WybpECUbZkPH/teCFjezYnNsa4adRcHn5L29VK8ZTb4FhgZlMIBcdkMysL/NxsYDVgY47Hm8JtOdUF6prZTDObY2bp4fbFQLqZlQ5fjroMODuXtQJgZj3NLMvMsrZt25aXQ0VOWfNalZjYtzV3p9XijayNtBmYwZRlW4IuSyRf5TY47gAeBi5x931APHBbPrx+HFAHuBToBjxvZhXcfQrwITALeA2YDRzOyxO7+0h3T3b35MTExHwoVSR3SsbH8ki7CxjfO4WKZRLo+fIC+ryqDaOk+MhtcLQAVrr7D2bWHfgdsPNnjtnMT0cJ1cNtOW0CJrj7QXdfB6wiFCS4+9/cvbG7tyF0WWxVLmsVKRSOblf74FV1mbLsW64cmME7n23Sku1S5OU2OIYD+8ysEfAAsAYY/TPHzAfqmFlNM0sAugITjukzntBog/AlqbrAWjOLNbNK4faGQENgSi5rFSk04mNj6HN5HT7s24palcvQ//XF3PZPLdkuRVtug+OQh/5M6gA84+7DgLInO8DdDwF9gMnAF8Ab7r7MzB41s/bhbpOB7Wa2HPgEeMjdtxO6FDY93D4S6B5+PszsfjPbRGgEs8TMRuXlhEWCUPuMsrx5T0v+dF0S89Z9z1UDM3h59ldaNFGKJMvNsNnMMoBJwO1Aa2ArsNjdL4xsefkjOTnZs7Kygi5DBICN3+/jt+98zvQvv6NpjYr8X6cLOS/xtKDLEvkvZrbA3ZOPbc/tiKMLcIDQ/RxbCP21/2Q+1icSNc6uWJrRtzflyc4NWbFlF+2GTOfZT1dr0UQpMnIVHOGwGAOUN7Nrgf3u/nNzHCJyAmbGDclnM+2BNK6odwZPTFpJx2dnsuzrn/vMiUjwcrvkyI3APOAG4EZgrpl1jmRhItHgjLIlGd79Yob/qglbdh6g/TMzeXLyCvYfzNOnz0UKVG7nOBYDbY7e2W1micA0d28U4fryheY4pCj4YV82f/vgC95csIlaiWV4olNDkmtUDLosiWK/dI4jJsdyIADb83CsiORChdIJPHlDI0bf3pQDB49ww4jZ/PHdpew5oEUTpXDJ7S//SWY22cxuNbNbgQ8I3dktIvkstW4iU/qnckuLGoyes562gzLJWKVlc6TwyNWlKgAz6wSkhB9Od/d3IlZVPtOlKimqFqz/nl+PW8KabXu5vkk1/nBtEhVKJwRdlkSJE12qynVwFGUKDinK9h88zDMfr+a5jDVUKB3Pox0a0K5BVcx+boFqkV/mlOY4zGy3me06ztduM9sVuXJF5KiS8bE82PZ83u2TQtXyJbl3zELueWUBW3ftD7o0iVInDQ53L+vu5Y7zVdbdy53sWBHJX/XPKs/4e1N4uF09Pl25jSsHZvDG/I1aNFEKnD4ZJVKExMXGcE/aeUzs25p6Z5bj128tofsLc9mwfV/QpUkUUXCIFEG1Ek9j7F3N+UvHBizeuJOrBmcwMnONli2RAqHgECmiYmKMHs3PZeqAVFrVrszfP1yhZUukQCg4RIq4M8uX4vmbkxl203+WLXlsopYtkchRcIgUA2bGNQ3PZNqAVDo1qcZzGWtIH5zJrDXfBV2aFEMKDpFipELpBJ7o3IgxdzbDgZuen8vDby1h576DQZcmxYiCQ6QYSqldmUl9U7k7rRZvLtjElYMymPj5N/roruQLBYdIMVUqIZZH2l3Au71TOKNsCXqNWcjdLy9gy07dOCi/jIJDpJhrUK087/ZO4ZF29chYtY02AzMYM3e99juXU6bgEIkCcbEx3J12HpP7pXJh9fL87ztL6TpyDmu27Qm6NCmCFBwiUaRG5TKMubMZTxzd73zwdJ75+EuyD+nGQck9BYdIlDEzbgzvd96mfhX+MWUV7Z+ZwaKNPwRdmhQRCg6RKHVG2ZIMu6kJz9+czA/7DvI/z87k0feWs1c7DsrPUHCIRLk2SVWYOiCV7s3O5cWZ67hqUCafrtz68wdK1FJwiAhlS8bzl44NePOeFpSMj+HWl+bT//VFfL83O+jSpBBScIjIv11SoyIf9m3N/VfU4f0lX3PlwAzGf7ZZNw7KTyg4ROQnSsTFMqBNXd6/rzXnVCxNv9cXcetL89m0Q3t+SIiCQ0SO6/yqZXmrV0v+dF0S87/6nqsGZfLijHUc1o2DUU/BISInFBtj3JpSkyn9U2lasyKPvr+c64fPYsWWXUGXJgFScIjIz6p+emleuvUShnRtzMbv93Ht0Bk8NWWl9vyIUgoOEckVM6ND42pMG5BG+0Zn8fTHq7l66HTmrfs+6NKkgCk4RCRPKpZJYGCXxvzr9qYcOHiEG0fM5n/f+Zxd+7XnR7RQcIjIKUmrm8iU/qnc0aomr83bQJuBGUxauiXosqQAKDhE5JSVKRHH769N4p17U6hYpgT3vLKAnqOz+Gbnj0GXJhGk4BCRX6zR2RWY0Ce050fml9toMzCT0bO/0kd3iykFh4jki/jwnh9T+qVx0TkV+MO7y+ikj+4WSxENDjNLN7OVZrbazB4+QZ8bzWy5mS0zs1dztD9uZkvDX11ytNc0s7nh53zdzBIieQ4ikjfnVCrN6NubMrhLYzaEP7r7xKQV+uhuMRKx4DCzWGAY0A5IArqZWdIxfeoAjwAp7l4f6BduvwZoAjQGmgEPmlm58GGPA4PcvTawA7gjUucgIqfGzOh4UTU+GpBGx4uq8eyna2g7OJOZq78LujTJB5EccTQFVrv7WnfPBsYCHY7pcxcwzN13ALj70bWck4BMdz/k7nuBJUC6mRlwOTAu3O9fQMcInoOI/AKnl0ngHzc04tU7m2HAr0bNZcAbWnW3qItkcFQDNuZ4vCncllNdoK6ZzTSzOWaWHm5fTCgoSptZZeAy4GygEvCDux86yXMCYGY9zSzLzLK2bduWT6ckIqeiZe3KTOqXSp/LajNhUWjV3Xc+26RVd4uooCfH44A6wKVAN+B5M6vg7lOAD4FZwGvAbCBPF0jdfaS7J7t7cmJiYv5WLSJ5VjI+lgfbns8H97fm3Eql6f/6Ym5+cR7rt+8NujTJo0gGx2ZCo4SjqofbctoETHD3g+6+DlhFKEhw97+5e2N3bwNY+HvbgQpmFneS5xSRQuz8qmUZd09L/tKhPp9t+IGrBmUy/NM1HDx8JOjSJJciGRzzgTrhT0ElAF2BCcf0GU9otEH4klRdYK2ZxZpZpXB7Q6AhMMVD49pPgM7h428B3o3gOYhIBMTGGD1a1GDagDQuPT+Rxyet4LqnZ7Bo4w9Blya5ELHgCM9D9AEmA18Ab7j7MjN71Mzah7tNBrab2XJCgfCQu28H4oHp4faRQPcc8xq/AQaY2WpCcx4vROocRCSyqpYvyYgeyTzX/WJ27Mvmf56dyZ8mLGPPgUM/f7AExqJhcio5OdmzsrKCLkNETmL3/oM8OXklL89ZT9VyJXm0QwPaJFUJuqyoZmYL3D352PagJ8dFRAAoWzKeRzs04K1eLSlXMp67RmfR65UFfLtrf9ClyTEUHCJSqDQ553Tev78VD7U9n49WbOXKpzJ4Zc56jmjdq0JDwSEihU58bAy9L6vN5H6pXFi9PL8bv5QbRsxm1be7gy5NUHCISCFWs3IZxtzZjH/c0Ig12/ZwzdDpDNSWtYFTcIhIoWZmdL64Oh8NSOPahmcx9OPVXD1kOrPXbA+6tKil4BCRIqHSaSUY1KUxL9/RlENHnG7Pz+HX4xbzwz6te1XQFBwiUqS0rpPI5H6p3JN2Hm8t3MyVAzN4d9FmrXtVgBQcIlLklEqI5eF29XivTyuqVShF37GLuPWl+Wz8fl/QpUUFBYeIFFlJZ5Xj7XtT+ON1Scz/6nvaDMpgRIbWvYo0BYeIFGmxMcZtKTWZOiCNVrUr838TV9D+mZl8tmFH0KUVWwoOESkWqlUoxfM3h9e92pvN9cNn8fvxS9m1/2DQpRU7Cg4RKTbMjPQGVZk6IJVbWtRgzNz1XPFUBh8s+UaT5/lIwSEixU7ZkvH8qX19xvdOoUq5EvR+dSG3/VOT5/lFwSEixVbD6hUYf28Kv782iXnrQpPnz2ny/BdTcIhIsRYXG8MdrWoybUAaresk8tjE0KZRC9Zr8vxUKThEJCqcFZ48H9HjYnb+eJDOz83if9/5nJ0/avI8rxQcIhJV2tavytQBadzWsiavzdvAFU9l8N7irzV5ngcKDhGJOqeViOMP1yUxoU8rzixfkvte+4xbXprPhu2aPM8NBYeIRK0G1cozvnfozvMF4TvPn/10tSbPf4aCQ0Si2tE7z6c9kMal5yfyxKSVXDt0BgvWfx90aYWWgkNEBDizfClG9Ejm+ZuT2b3/IJ2Gz+a373zOzn2aPD+WgkNEJIc2SVWYOiCNO1vVZOy8DVwx8FMt234MBYeIyDHKlIjjd9eGJs/PCi/bfvOL81i/fW/QpRUKCg4RkRNoUK0879ybwp/b1+ezDT9w1aBMhn2ymuxD0T15ruAQETmJ2BjjlpY1mDYgjcvrncGTk1dyzdDpzP8qeifPFRwiIrlQtXxJhne/mFE3J7Mv+zA3PDebR95eEpV7nis4RETy4MqkKkzpn0rP1Fq8kbUpKvc8V3CIiORRmRJx/PbqC5jQJ4Vqp5em79hF9HhhHl99Fx2T5woOEZFTVP+s8rzdqyV/6VCfxRt/4KrBmTz90ZfFfvJcwSEi8gvExhg9WtRg2gNptLmgCk9NXcXVQ6czd+32oEuLGAWHiEg+qFKuJMN+1YSXbr2EH7MP02XkHB58czHb9xwIurR8p+AQEclHl9U7g2kD0uh16XmM/2wzVwzMYOy8DRw5UnwmzxUcIiL5rFRCLL9Jr8eHfVtT94yyPPz259wwYjYrtuwKurR8oeAQEYmQulXK8vrdzXmyc0PWbtvDNUNn8PcPv2DvgUNBl/aLKDhERCLIzLgh+Ww+fuBSbri4OiMz19JmYAZTlm0JurRTpuAQESkAp5dJ4LFODRl3TwvKloyn58sLuPNf89m0o+jtOhjR4DCzdDNbaWarzezhE/S50cyWm9kyM3s1R/sT4bYvzGyomVm4vYuZLQl/7/FI1i8ikt+Sa1Tk/ftb8dur6zFz9XbaDMzkuYw1RWrXwYgFh5nFAsOAdkAS0M3Mko7pUwd4BEhx9/pAv3B7SyAFaAg0AC4B0sysEvAkcEW4f1UzuyJS5yAiEgnxsTH0TD2PaQ+k0apOZR6buKJILZwYyRFHU2C1u69192xgLNDhmD53AcPcfQeAu28NtztQEkgASgDxwLdALeBLd98W7jcN6BTBcxARiZhqFUrx/M2hXQf3HggtnPjrcYv5fm/hXjgxksFRDdiY4/GmcFtOdYG6ZjbTzOaYWTqAu88GPgG+CX9NdvcvgNXA+WZWw8zigI7A2cd7cTPraWZZZpa1bdu243URESkUQrsOpnJ3Wi3eXriZK576lDeyNhbaez+CnhyPA+oAlwLdgOfNrIKZ1QYuAKoTCpvLzax1eGTSC3gdmA58BRw+3hO7+0h3T3b35MTExIifiIjIL1E6IY5H2l3AB/e35rzE0/j1uCV0GTmblVt2B13af4lkcGzmp6OB6uG2nDYBE9z9oLuvA1YRCpL/Aea4+x533wNMBFoAuPt77t7M3VsAK8PHiIgUC+dXLcsbd7fgiU4N+XLrHq4ZOp3HJq5gX3bhufcjksExH6hjZjXNLAHoCkw4ps94QqMNzKwyoUtXa4ENhCbD48wsHkgDvgj3OyP839OBe4FRETwHEZECFxNj3HhJ6N6P65tU47mMNbQZmMm05d8GXRoQweBw90NAH2AyoV/6b7j7MjN71Mzah7tNBrab2XJCcxoPuft2YBywBvgcWAwsdvf3wscMCfefCTzm7hpxiEixVLFMAk90bsQbd7egTIlY7hydRc/RWWz+4cdA67Jo2LUqOTnZs7Kygi5DROSUZR86wgsz1jHko1XEmNHvyjrcllKT+NjIXTgyswXunnxse9CT4yIikgsJcTH0uvQ8pvZPo0WtSvz9wxVc9/QMFqwv+Hs/FBwiIkXI2RVLM+qWZEb0uJhdPx6k0/DZPPzWEnYU4L0fCg4RkSLGzGhbvypTB6TRM7UWby7YxBUDMxi3YBMFMf2g4BARKaLKlIjjt1dfwPv3taJGpdI8+OZiuo6cw5ffRvbeDwWHiEgRd8GZ5Rh3T0seu/5CVmzZTbsh03li0gp+zD7u/dG/mIJDRKQYiIkxujY9h48fSKND42o8++ka2gzKiMid5woOEZFipNJpJXjqxkaM7dmcmpXLUP30Uvn+GnH5/owiIhK45rUq0bxWpYg8t0YcIiKSJwoOERHJEwWHiIjkiYJDRETyRMEhIiJ5ouAQEZE8UXCIiEieKDhERCRPomIjJzPbBqw/xcMrA9/lYzlFnd6P/9B78VN6P36qOLwf57p74rGNUREcv4SZZR1vB6xopffjP/Re/JTej58qzu+HLlWJiEieKDhERCRPFBw/b2TQBRQyej/+Q+/FT+n9+Kli+35ojkNERPJEIw4REckTBYeIiOSJguMkzCzdzFaa2WozezjoeoJiZmeb2SdmttzMlplZ36BrKgzMLNbMPjOz94OuJWhmVsHMxpnZCjP7wsxaBF1TUMysf/jnZKmZvWZmJYOuKb8pOE7AzGKBYUA7IAnoZmZJwVYVmEPAA+6eBDQHekfxe5FTX+CLoIsoJIYAk9y9HtCIKH1fzKwacD+Q7O4NgFiga7BV5T8Fx4k1BVa7+1p3zwbGAh0CrikQ7v6Nuy8M/3s3oV8K1YKtKlhmVh24BhgVdC1BM7PyQCrwAoC7Z7v7D4EWFaw4oJSZxQGlga8DriffKThOrBqwMcfjTUT5L0sAM6sBXATMDbiUoA0Gfg0cCbiOwqAmsA14KXzpbpSZlQm6qCC4+2bgH8AG4Btgp7tPCbaq/KfgkFwzs9OAt4B+7r4r6HqCYmbXAlvdfUHQtRQScUATYLi7XwTsBaJyTtDMTid0ZaImcBZQxsy6B1tV/lNwnNhm4Owcj6uH26KSmcUTCo0x7v520PUELAVob2ZfEbqEebmZvRJsSYHaBGxy96Oj0HGEgiQaXQmsc/dt7n4QeBtoGXBN+U7BcWLzgTpmVtPMEghNcE0IuKZAmJkRun79hbsPDLqeoLn7I+5e3d1rEPr/4mN3L3Z/VeaWu28BNprZ+eGmK4DlAZYUpA1AczMrHf65uYJi+EGBuKALKKzc/ZCZ9QEmE/pkxIvuvizgsoKSAvQAPjezReG237r7h8GVJIXMfcCY8B9Za4HbAq4nEO4+18zGAQsJfRrxM4rh0iNackRERPJEl6pERCRPFBwiIpInCg4REckTBYeIiOSJgkNERPJEwSFSyJnZpVqBVwoTBYeIiOSJgkMkn5hZdzObZ2aLzGxEeL+OPWY2KLw/w0dmlhju29jM5pjZEjN7J7zGEWZW28ymmdliM1toZueFn/60HPtdjAnflSwSCAWHSD4wswuALkCKuzcGDgO/AsoAWe5eH8gA/hg+ZDTwG3dvCHyeo30MMMzdGxFa4+ibcPtFQD9Ce8PUInQ3v0ggtOSISP64ArgYmB8eDJQCthJadv31cJ9XgLfD+1dUcPeMcPu/gDfNrCxQzd3fAXD3/QDh55vn7pvCjxcBNYAZET8rkeNQcIjkDwP+5e6P/KTR7PfH9DvVNX4O5Pj3YfSzKwHSpSqR/PER0NnMzgAws4pmdi6hn7HO4T43ATPcfSeww8xah9t7ABnh3RU3mVnH8HOUMLPSBXkSIrmhv1pE8oG7Lzez3wFTzCwGOAj0JrSpUdPw97YSmgcBuAV4LhwMOVeT7QGMMLNHw89xQwGehkiuaHVckQgysz3uflrQdYjkJ12qEhGRPNGIQ0RE8kQjDhERyRMFh4iI5ImCQ0RE8kTBISIieaLgEBGRPPl/ltI07io0hAwAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# Load the data\n",
    "x1, _ = breast_cancer(party_id=1,train=True)\n",
    "x2, y = breast_cancer(party_id=2,train=True)\n",
    "\n",
    "# Hyperparameter\n",
    "W = jnp.zeros((30,))\n",
    "b = 0.0\n",
    "epochs = 10\n",
    "learning_rate = 1e-2\n",
    "\n",
    "# Train the model\n",
    "losses, W, b = fit(W, b, x1, x2, y, epochs=10, learning_rate=1e-2)\n",
    "\n",
    "# Plot the loss\n",
    "plot_losses(losses)\n",
    "\n",
    "# Validate the model\n",
    "X_test, y_test = breast_cancer(train=False)\n",
    "auc=validate_model(W,b, X_test, y_test)\n",
    "print(f'auc={auc}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb11163f",
   "metadata": {},
   "source": [
    "Just remember the plot and AUC here since we would like to do the similar thing with SPU!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5c4d4b1",
   "metadata": {},
   "source": [
    "## Train a Model with SPU\n",
    "\n",
    "At this part, we are going to show you how to do the similar trainning with MPC securely!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fe0ded7-d861-4506-9b87-4342629a8c5e",
   "metadata": {},
   "source": [
    "### Init the Environment\n",
    "\n",
    "We are going to init three virtual devices on our physical environment.\n",
    "- alice, bob：Two PYU devices for local plaintext computation.\n",
    "- spu：SPU device consists with alice and bob for MPC secure computation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "14c4f276-7eb0-4dd9-a39b-06b588bcc5bf",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-08-24 18:08:23.975294: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n"
     ]
    }
   ],
   "source": [
    "import secretflow as sf\n",
    "\n",
    "# In case you have a running secretflow runtime already.\n",
    "sf.shutdown()\n",
    "\n",
    "sf.init(['alice', 'bob'], num_cpus=8, log_to_driver=True)\n",
    "\n",
    "alice, bob = sf.PYU('alice'), sf.PYU('bob')\n",
    "spu = sf.SPU(sf.utils.testing.cluster_def(['alice', 'bob']))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cecb025b",
   "metadata": {},
   "source": [
    "### Load the Dataset\n",
    "\n",
    "we instruct alice and bob to load the train subset repectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "149864cc-ea35-4f3e-bb7a-2247ef835ea8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<secretflow.device.device.pyu.PYUObject at 0x7f5dd0790490>,\n",
       " <secretflow.device.device.pyu.PYUObject at 0x7f5f3c1b5df0>,\n",
       " <secretflow.device.device.pyu.PYUObject at 0x7f5f3c1b5d00>)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x1, _ = alice(breast_cancer)(party_id=1)\n",
    "x2, y = bob(breast_cancer)(party_id=2)\n",
    "\n",
    "x1, x2, y\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "019e3f74-cfa8-458b-b3ca-6d3431d0e64d",
   "metadata": {},
   "source": [
    "Before trainning, we need to pass hyperparamters and all data to SPU device. SecretFlow provides two methods:\n",
    "- secretflow.to: transfer a PythonObject or DeviceObject to a specific device.\n",
    "- DeviceObject.to: transfer the DeviceObject to a specific device."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a899ee65-a963-4f75-b5bf-e97ed885b52f",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "device = spu\n",
    "\n",
    "W = jnp.zeros((30,))\n",
    "b = 0.0\n",
    "\n",
    "W_, b_, x1_, x2_, y_ = (\n",
    "    sf.to(device, W),\n",
    "    sf.to(device, b),\n",
    "    x1.to(device),\n",
    "    x2.to(device),\n",
    "    y.to(device),\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71fabe33-8a83-4cd7-97ef-66743ce4d492",
   "metadata": {},
   "source": [
    "### Train the model\n",
    "\n",
    "Now we are ready to train a LR model with SPU. After trainning, losses and model are SPUObjects which are still secret."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d2b3a1a2-f12d-4fe7-bea5-5a6e4faf01ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(<secretflow.device.device.spu.SPUObject at 0x7f5f3c1c5430>,\n",
       " <secretflow.device.device.spu.SPUObject at 0x7f5f3c1c52e0>,\n",
       " <secretflow.device.device.spu.SPUObject at 0x7f5f3c1c53a0>)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "losses, W_, b_ = device(\n",
    "    fit,\n",
    "    static_argnames=['epochs'],\n",
    "    num_returns_policy=sf.device.SPUCompilerNumReturnsPolicy.FROM_USER,\n",
    "    user_specified_num_returns=3,\n",
    ")(W_, b_, x1_, x2_, y_, epochs=10, learning_rate=1e-2)\n",
    "\n",
    "losses, W_, b_\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea3a5d7b-f16d-4330-a96f-5a4f6b0a2996",
   "metadata": {},
   "source": [
    "### Reveal the result\n",
    "\n",
    "In order to check losses and model, we need to convert SPUObject(secret) to Python object(plaintest). SecretFlow provide `sf.reveal` to convert any DeviceObject to Python object.\n",
    "\n",
    "> Be care with `sf.reveal`，since it may result in secret leak。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8a725cc8-21f4-4e02-ae77-a10f2dcd1187",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=859106)\u001b[0m 2022-08-24 18:08:29.753681: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n",
      "\u001b[2m\u001b[36m(_run pid=859102)\u001b[0m 2022-08-24 18:08:29.753681: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n",
      "\u001b[2m\u001b[36m(pid=859101)\u001b[0m 2022-08-24 18:08:30.180754: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n",
      "\u001b[2m\u001b[36m(pid=859100)\u001b[0m 2022-08-24 18:08:30.180754: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n",
      "\u001b[2m\u001b[36m(pid=859103)\u001b[0m 2022-08-24 18:08:30.180754: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n",
      "\u001b[2m\u001b[36m(pid=859105)\u001b[0m 2022-08-24 18:08:30.180754: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/rh-ruby25/root/usr/local/lib64:/opt/rh/rh-ruby25/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib64:/opt/rh/devtoolset-11/root/usr/lib:/opt/rh/devtoolset-11/root/usr/lib64/dyninst:/opt/rh/devtoolset-11/root/usr/lib/dyninst\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859104)\u001b[0m I0824 18:08:31.343399 859104 external/com_github_brpc_brpc/src/brpc/server.cpp:1066] Server[yasl::link::internal::ReceiverServiceImpl] is serving on port=28315.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859104)\u001b[0m I0824 18:08:31.343465 859104 external/com_github_brpc_brpc/src/brpc/server.cpp:1069] Check out http://k69b13338.eu95sqa:28315 in web browser.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859107)\u001b[0m I0824 18:08:31.362519 859107 external/com_github_brpc_brpc/src/brpc/server.cpp:1066] Server[yasl::link::internal::ReceiverServiceImpl] is serving on port=27815.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859107)\u001b[0m I0824 18:08:31.362593 859107 external/com_github_brpc_brpc/src/brpc/server.cpp:1069] Check out http://k69b13338.eu95sqa:27815 in web browser.\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859104)\u001b[0m I0824 18:08:31.444237 859517 external/com_github_brpc_brpc/src/brpc/socket.cpp:2202] Checking Socket{id=0 addr=127.0.0.1:27815} (0x55b347c82480)\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859104)\u001b[0m I0824 18:08:31.444437 859517 external/com_github_brpc_brpc/src/brpc/socket.cpp:2262] Revived Socket{id=0 addr=127.0.0.1:27815} (0x55b347c82480) (Connectable)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=859102)\u001b[0m [2022-08-24 18:08:31.512] [info] [thread_pool.cc:30] Create a fixed thread pool with size 63\n",
      "\u001b[2m\u001b[36m(_run pid=859106)\u001b[0m [2022-08-24 18:08:31.500] [info] [thread_pool.cc:30] Create a fixed thread pool with size 63\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(_run pid=859106)\u001b[0m 2022-08-24 18:08:31,500,500 WARNING [xla_bridge.py:backends:265] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n",
      "\u001b[2m\u001b[36m(_run pid=859102)\u001b[0m 2022-08-24 18:08:31,511,511 WARNING [xla_bridge.py:backends:265] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n",
      "\u001b[2m\u001b[36m(_run pid=859105)\u001b[0m 2022-08-24 18:08:32,005,5 WARNING [xla_bridge.py:backends:265] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2m\u001b[36m(SPURuntime pid=859104)\u001b[0m [2022-08-24 18:08:32.504] [info] [thread_pool.cc:30] Create a fixed thread pool with size 63\n",
      "\u001b[2m\u001b[36m(SPURuntime pid=859107)\u001b[0m [2022-08-24 18:08:32.507] [info] [thread_pool.cc:30] Create a fixed thread pool with size 63\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmA0lEQVR4nO3dd3hVVbrH8e+bQugiEHQoUgREQGpEIEBQQWIDu6CiYy8oAo6FuTN3HO84o+MINhCxjF1UbKhIcxwCAZSAgnQpKiAIIiA9BN77x9k6gaEkmJN9kvP7PE8eOSt7n7z7PIYfa6+91jJ3R0REpKASwi5ARERKFgWHiIgUioJDREQKRcEhIiKFouAQEZFCSQq7gOJQvXp1r1evXthliIiUKLNmzfrB3VP3b4+L4KhXrx45OTlhlyEiUqKY2TcHatetKhERKRQFh4iIFIqCQ0RECkXBISIihaLgEBGRQlFwiIhIoSg4RESkUBQch/DB3O949/PVaOl5EZH/UHAcwluzVjHg9S+45ZXZbNi6K+xyRERigoLjEJ656mTuzmzCxwvX0eORLCYu+D7skkREQqfgOITEBOPmrscz5rZ0UiuV5foXc7jzzTls2bk77NJEREKj4CiAJsdW5r1+6fQ79Xjemr2KzEemMG3ZD2GXJSISiqgGh5llmtliM1tqZvcc5JhLzGyBmc03s1fztT9oZvOCr0vztT9rZnPMbK6ZjTazitG8hp+VSUrgzh5NGH1zR8okJXDZ059y75j57MjdUxw/XkQkZkQtOMwsERgGnAk0BfqYWdP9jmkEDAbS3b0ZMCBoPxtoA7QCTgF+Z2aVg9MGuntLd28BfAvcGq1rOJA2xx3N2P6d+W3Hejw/7WvOfnwKX6zcVJwliIiEKpo9jnbAUndf7u65wCig137HXA8Mc/eNAO6+LmhvCmS5e567bwPmApnBMT8BmJkB5YBif1a2XJlE7u3ZjFeuO4WduXu48MlpPDxhMbl5e4u7FBGRYhfN4KgFrMz3elXQll9joLGZZZvZDDPLDNrnAJlmVt7MqgOnAnV+PsnM/gmsBZoAjx/oh5vZDWaWY2Y569evL5or2k96w+qMG9iF81rV4vF/LeX84dksXrslKj9LRCRWhD04ngQ0AroCfYCnzayKu08AxgLTgNeA6cAvgwnufjVQE1gIXMoBuPtId09z97TU1P/awKrIVC6bzMOXtOSpvm1Zu3kn5z4+lacmL2PPXk0aFJHSKZrBsZp8vQSgdtCW3ypgjLvvdvcVwBIiQYK73+/urdy9O2DB937h7nuI3P66MEr1F0qPZscyfmAXTm2Syt8+WkTvkdP5ZsO2sMsSESly0QyOmUAjM6tvZmWA3sCY/Y55l0hvg+CWVGNguZklmlm1oL0F0AKYYBENg3YDegKLongNhVK9YgojrmjLkEtasmjtFs58dAovz/hGS5aISKkStT3H3T3PzG4FxgOJwHPuPt/M7gNy3H1M8L0zzGwBkVtRd7r7BjMrC0yJZAM/AVcE75cAvBA8YWVExkJujtY1HAkz44I2tWnfoBp3jZ7LH96dx8QF3/PghS049qiyYZcnIvKrWTz8azgtLc1zcnKK/efu3eu8/Ok3/HXsQsokJvB/5zWnZ8uaBIEoIhLTzGyWu6ft3x724HiplpBgXNmhHh/d3oWGNSpy+6gv6PfqbH7clht2aSIiR0zBUQzqV6/Amzd15K7ME5i44HvOGJrFxwu1YKKIlEwKjmKSmGDc0rUh7/XrRPWKZbj2hRzuGq0FE0Wk5FFwFLOmNSvz3q3p3NL1eEbPiiyYOH3ZhrDLEhEpMAVHCFKSErkrswlv3tSR5ESjz9MzuO/9BezcrQUTRST2KThC1Lbu0Yy9vTNXdqjLc9krOPuxKczRgokiEuMUHCErXyaJ+3o15+VrT2F77h4ueHIaQyYsZvceLZgoIrFJwREjOjWqzrgBXejVqiaPBQsmLvleCyaKSOxRcMSQo8olM+SSVoy4oi1rNu3knMenMjJLCyaKSGxRcMSgzOaRBRO7Nk7lr2MX0WfkDL7dsD3sskREAAVHzKpeMYWn+rbl4YtbsnDNT2Q+msVLWjBRRGKAgiOGmRkXtq3N+IFdaFv3aP747jz6PvsZqzftCLs0EYljCo4SoGaVcrx4TTvuP785s7/dSObQLN6YuVK9DxEJhYKjhDAzLj+lLuMHdKFpzcrc9dZcrnl+Jt//tDPs0kQkzig4Spg6Vcvz2vXt+dO5TZm+fANnDM3i3c9Xq/chIsVGwVECJSQYV6fXZ2z/zhyfWoEBr3/BTS/PYv2WXWGXJiJxQMFRgjVIrcibN3Vk8JlN+GTxeno8ksWHc9eEXZaIlHIKjhIuMcG4MeN4PrytE7WPLke/V2dz66uz2ajNokQkShQcpUSjYyrx9s0duaN7Y8bPX0v3oVlMXKDNokSk6Ck4SpGkxARuO70R7/XrRGqlFK5/MYdBb3zB5h3aLEpEio6CoxRqWrMy7/VLp/9pDXnvi+/oMTSLyUvWh12WiJQSCo5SqkxSAoPOOIF3bulIxbJJXPXcZwx+ey5bd+WFXZqIlHAKjlKuRe0qfHBbJ27s0oBRM1fSY2gW05b9EHZZIlKCKTjiQNnkRAafdSKjb+pAmaQELnv6U/703jy256r3ISKFp+CII23rVmVs/878tmM9Xpj+DWc9OoWcr38MuywRKWEUHHGmXJlE7u3ZjFE3tGePOxc/NZ37P1zAzt17wi5NREoIBUecat+gGuNu78Jl7Y7j6SkrOPuxKXyxclPYZYlICaDgiGMVUpK4//yTeOnadmzP3cMFw7N5aPwiduWp9yEiB6fgEDo3SmX8wC5c2KY2wz5ZRq8nspm3enPYZYlIjFJwCACVyybz0MUtefaqNDZsy+W8Ydk8MmkJu/fsDbs0EYkxCg7Zx+knHsPEgV04u8VveGTSV5w/PJvFa7eEXZaIxBAFh/yXKuXL8Gjv1oy4og1rNu3k3MenMvzfS8lT70NEUHDIIWQ2/w0TBnbh9BNr8Pdxi7loxHSWrtsadlkiEjIFhxxStYopDL+8DY/1ac3XG7Zx1mNTGJm1jD17tVWtSLxScMhhmRk9W9ZkwsAuZDRO5a9jF3HRiGnqfYjEKQWHFFiNSmUZ2bctj/ZuxYof1PsQiVdRDQ4zyzSzxWa21MzuOcgxl5jZAjObb2av5mt/0MzmBV+X5mt/JXjPeWb2nJklR/MaZF9mRq9WtdT7EIljUQsOM0sEhgFnAk2BPmbWdL9jGgGDgXR3bwYMCNrPBtoArYBTgN+ZWeXgtFeAJsBJQDngumhdgxyceh8i8SuaPY52wFJ3X+7uucAooNd+x1wPDHP3jQDuvi5obwpkuXueu28D5gKZwTFjPQB8BtSO4jXIIaj3IRKfohkctYCV+V6vCtryaww0NrNsM5thZplB+xwg08zKm1l14FSgTv4Tg1tUfYFxB/rhZnaDmeWYWc769do2NZoO1Pt4arJ6HyKlVdiD40lAI6Ar0Ad42syquPsEYCwwDXgNmA7sv/LecCK9kikHemN3H+nuae6elpqaGq36JbB/7+NvH6n3IVJaRTM4VrNvL6F20JbfKmCMu+929xXAEiJBgrvf7+6t3L07YMH3ADCzPwGpwKAo1i9HQL0PkdIvmsExE2hkZvXNrAzQGxiz3zHvEultENySagwsN7NEM6sWtLcAWgATgtfXAT2APu6uNTBiUP7eR1f1PkRKnagFh7vnAbcC44GFwBvuPt/M7jOznsFh44ENZrYA+AS40903AMnAlKB9JHBF8H4AI4BjgOlm9oWZ/W+0rkF+nRqVyvKUeh8ipY5FHk4q3dLS0jwnJyfsMuLaui07+cM785iw4HtaH1eFhy5qScMaFcMuS0QOwcxmuXva/u1hD45LnFDvQ6T0UHBIsdHYh0jpoOCQYqfeh0jJpuCQUByo93Hhk+p9iJQECg4JVf7ex8/7faj3IRLbFBwSup97HxMHZnDqCep9iMQ6BYfEjNRKKYy4Qr0PkVin4JCYot6HSOxTcEhMUu9DJHYpOCRmqfchEpsUHBLzDtT7ePLfy8jbozUuRcKg4JASIX/v47QTavDguEVc8OQ0Fq39KezSROKOgkNKlNRKKYzo25Zhl7Vh9cYdnPv4VB6d9BW71fsQKTYKDimRzm7xGyYOyuDM5r9h6KQl9Hwim3mrN4ddlkhcUHBIiVW1Qhke69OakX3bsmHrLnoNy+ah8YvYlbf/LsMiUpQUHFLindHsWCYOzOCC1rUY9skyzn5sKp9/uzHsskRKLQWHlApHlU/moYtb8vzVJ7N9Vx4XPjmN+z9cwM7d6n2IFDUFh5QqXU+owfiBXejd7jienrKCMx+dwmcrfgy7LJFSRcEhpU6lssn89fyTePW6U8jbu5dLR07n3jHz2bYr7/Ani8hhKTik1OrYsDrjbu/CVR3q8cL0r+nxSBbZS38IuyyREk/BIaVahZQk7u3ZjDdu7ECZxAQuf+ZTBr/9JT/t3B12aSIlloJD4sLJ9aoy9vbO3NilAa/P/JYeQ7P4ZPG6sMsSKZEUHBI3yiYnMvisE3n7lnQqpiRx9T9ncscbc9i0PTfs0kRKFAWHxJ1WdarwQf9O3HZaQ979YjXdh2Yxfv7asMsSKTEUHBKXUpISueOME3ivXzrVK6Zw40uzuO21z9mwdVfYpYnEPAWHxLXmtY5izK3p3NG9MePmreGMoVl8MPc73LVhlMjBKDgk7iUnJnDb6Y344LbO1D66HLe++jk3vTyLdVt2hl2aSExScIgETji2Em/d3JF7zmzCJ4vX031IFm/PXqXeh8h+FBwi+SQlJnBTxvF8dHtnGtaoyKA35nDN8zNZs3lH2KWJxAwFh8gBHJ9akTdu7MCfzm3KjOU/csaQLEZ99q16HyIoOEQOKjHBuDq9PuMGdKZZrcrc8/aX9H32M1b+uD3s0kRCpeAQOYy61Srw6nXt+ct5zfn82430eCSLF6d/zd696n1IfFJwiBRAQoJxRfu6TBiUQVq9qvzve/Pp/fQMVvywLezSRIqdgkOkEGpVKccLV5/M3y9qwaI1P5H5SBYjJi8jb8/esEsTKTYKDpFCMjMuSavDpEEZdD0hlQc+WsT5w6ex4Lufwi5NpFgUKDjM7HYzq2wRz5rZbDM7I9rFicSyGpXLMuKKtgy/vA1rNu+g5xNTeXjCYnblabtaKd0K2uO4xt1/As4Ajgb6Ag8c7iQzyzSzxWa21MzuOcgxl5jZAjObb2av5mt/0MzmBV+X5mu/NXg/N7PqBaxfJCrMjLNO+g2TBmXQq1UtHv/XUs5+bCqzvtF2tVJ6FTQ4LPjvWcBL7j4/X9uBTzBLBIYBZwJNgT5m1nS/YxoBg4F0d28GDAjazwbaAK2AU4DfmVnl4LRsoBvwTQFrF4m6KuXL8PAlLXnhmnbsyN3DRSO0Xa2UXgUNjllmNoFIcIw3s0rA4UYD2wFL3X25u+cCo4Be+x1zPTDM3TcCuPvPO+s0BbLcPc/dtwFzgczgmM/d/esC1i1SrDIapzJ+YBeubF+XF6Z/zRlDs8hasj7sskSKVEGD41rgHuBkd98OJANXH+acWsDKfK9XBW35NQYam1m2mc0ws8ygfQ6QaWblg9tRpwJ1ClgrAGZ2g5nlmFnO+vX6xZXiUzEliT/3as4bN3YgJTmBK5/7jN+9OYfN27VdrZQOBQ2ODsBid99kZlcAfwA2F8HPTwIaAV2BPsDTZlbF3ScAY4FpwGvAdKBQI47uPtLd09w9LTU1tQhKFSmck+tVZWz/zvQ79Xje+Xw13YZOZty8NWGXJfKrFTQ4ngS2m1lL4A5gGfDiYc5Zzb69hNpBW36rgDHuvtvdVwBLiAQJ7n6/u7dy9+5ExlOWFLBWkZhRNjmRO3s0Ycyt6dSolMJNL8/mZi3ZLiVcQYMjzyOru/UCnnD3YUClw5wzE2hkZvXNrAzQGxiz3zHvEultENySagwsN7NEM6sWtLcAWgATClirSMxpVvMo3u2Xzt2ZTfh40Tq6D8nizZyVWjRRSqSCBscWMxtM5DHcD80sgcg4x0G5ex5wKzAeWAi84e7zzew+M+sZHDYe2GBmC4BPgDvdfUPw3lOC9pHAFcH7YWb9zWwVkR7MXDN7pjAXLBKW5MQEbu4aWbK98TEVuXP0XK58TosmSsljBfkXj5kdC1wGzHT3KWZ2HNDV3Q93uyompKWleU5OTthliPxi717nlU+/4YGPFuHAXT1OoG+HeiQmHPIpd5FiZWaz3D1t//YC9TjcfS3wCnCUmZ0D7CwpoSESixISjL4d6jFhUAYn16vKve8v4JKnprN03ZawSxM5rIIuOXIJ8BlwMXAJ8KmZXRTNwkTiQa0q5Xj+6pMZemlLlq3fylmPTuWJf33Fbi2aKDGsoLeq5gDdf56gZ2apwCR3bxnl+oqEblVJSfDD1l38acx8Ppy7hibHVuKhi1pyUu2jwi5L4tivulUFJOSb1Q2woRDnikgBVK+YwrDL2vBU37b8uC2X84Zn87ePFrJztxZNlNiSVMDjxpnZeCKT8QAuJTJBT0SKWI9mx9K+QTX+NnYhT01ezoT53/PABSdxSoNqYZcmAhTwVhWAmV0IpAcvp7j7O1GrqojpVpWUVNlLf+Cet+ey8scdXNH+OO7ObEKlsod8El6kyBzsVlWBg6MkU3BISbY9N48hE5bwXPYKjqlclvvPb85pTY4JuyyJA0c0xmFmW8zspwN8bTEzbXcmUgzKl0niD+c05a2bO1IxJYlrns9hwKjP+XFbbtilSZw6ZHC4eyV3r3yAr0ruXvlQ54pI0Wp93NF80L8Tt5/eiA+/XEO3IZMZM+c7LVsixU5PRomUIClJiQzs3pj3b+tEnaPL0f+1z7n+xRzWbN4RdmkSRxQcIiVQk2Mr8/Yt6fzPWScydekPdB+SxUvTv2bvXvU+JPoUHCIlVGKCcX2XBkwYkEGrOlX443vztWyJFAsFh0gJd1y18rx0bTv+cXFLvloXWbbk0UlfkZunZUskOhQcIqWAmXFR29pMGpRBj+bHMnTSEs55fAqzvtkYdmlSCik4REqR1EopPN6nNc9elcaWnXlcNGIa946Zz9ZdeWGXJqWIgkOkFDr9xGOYOCiDK9vX5YXpX9NjaBafLFp3+BNFCkDBIVJKVUxJ4s+9mjP6po6UL5PI1c/PpP9rn/PD1l1hlyYlnIJDpJRrWzcycXBAt0Z8NC8ycfCtWas0cVCOmIJDJA6kJCUyoFtjxvbvTIPqFbjjzTna71yOmIJDJI40OqYSo2/qyH29mjH7m42cMTSLZ6YsZ48mDkohKDhE4kxCgnFlh3pMHJRBx+Or8ZcPF3LB8GwWfKd1S6VgFBwicapmlXI8c1Uaj/dpzepNO+j5xFT+Pm6RdhyUw1JwiMQxM+PcljWZNCiD81rXYvi/l3Hmo1OYsXxD2KVJDFNwiAhVypfhHxe35OVrTyFv7156j5zB4LfnsnnH7rBLkxik4BCRX3RqVJ0JAzK4sUsDXp+5ku5DJjNu3tqwy5IYo+AQkX2UK5PI4LNO5L1+naheMYWbXp7FTS/N4vufdoZdmsQIBYeIHNBJtY/ivVvTuTuzCZ8sXke3IZN59dNvteeHKDhE5OCSExO4uevxjBvQhWY1K/P7d76kz9MzWL5+a9ilSYgUHCJyWPWrV+C169vz4IUnsXDNT2Q+OoVhnyxl9x7t+RGPFBwiUiBmxqUnH8ekOzLofuIxPDR+Mec+PpU5KzeFXZoUMwWHiBRKjUplGXZ5G0b2bcvG7bmcPzyb//tgAdtztedHvFBwiMgROaPZsUwclEGfdsfx7NQVdB+SxSeLtedHPFBwiMgRq1w2mfvPP4k3buxA2eQErv5nZM+P9Vu050dppuAQkV+tXf2qjL29MwO6NWLcvLV0GzKZ12d+qz0/SikFh4gUiV/2/Li9MyccU4m73/qS3iNnsEyP7pY6UQ0OM8s0s8VmttTM7jnIMZeY2QIzm29mr+Zrf9DM5gVfl+Zrr29mnwbv+bqZlYnmNYhI4TSsUZFRN7TngQsij+6e+cgUHvv4K3Lz9OhuaRG14DCzRGAYcCbQFOhjZk33O6YRMBhId/dmwICg/WygDdAKOAX4nZlVDk57EBjq7g2BjcC10boGETkyCQlG73aRR3fPaHYMQyYu4ezHppDz9Y9hlyZFIJo9jnbAUndf7u65wCig137HXA8Mc/eNAO7+8yMZTYEsd89z923AXCDTzAw4DRgdHPcCcF4Ur0FEfoUalcryxGVt+OdvT2Z77h4uGjGd37/zpVbdLeGiGRy1gJX5Xq8K2vJrDDQ2s2wzm2FmmUH7HCJBUd7MqgOnAnWAasAmd887xHsCYGY3mFmOmeWsX7++iC5JRI7EqU1qMGFgF67rVJ9Rn31LtyGTGfvlGg2el1BhD44nAY2ArkAf4Gkzq+LuE4CxwDTgNWA6UKhtydx9pLunuXtaampq0VYtIoVWISWJP5zTlPf6daJGpRRueWU217+Yw3ebdoRdmhRSNINjNZFews9qB235rQLGuPtud18BLCESJLj7/e7eyt27AxZ8bwNQxcySDvGeIhLDTqp9FO/1S+d/zjqR7KUb6D5kMs9NXcEerbpbYkQzOGYCjYKnoMoAvYEx+x3zLpHeBsEtqcbAcjNLNLNqQXsLoAUwwSP92k+Ai4LzrwLei+I1iEgUJCUmcH2XBkwY2IW0elW574MFXDA8m/nfbQ67NCmAqAVHMA5xKzAeWAi84e7zzew+M+sZHDYe2GBmC4gEwp3uvgFIBqYE7SOBK/KNa9wNDDKzpUTGPJ6N1jWISHTVqVqe568+mcf6tGb1ph30fCKbv320kB25hbozLcXM4mFwKi0tzXNycsIuQ0QOYdP2XP42dhGv56ykTtVy3H/eSXRprPHJMJnZLHdP27897MFxEREAqpQvw4MXtWDUDe1JTkjgyuc+Y8Coz/lhq9a9ijUKDhGJKe0bVGPs7Z3pf3ojPvxyDd2GTObNnJV6dDeGKDhEJOaUTU5kUPfGjO3fmYapFblz9Fwuf+ZTVvywLezSBAWHiMSwRsdU4o0bO3D/+c35cvVmejySxbBPlmrdq5ApOEQkpiUkGJefUpePB+27Ze2sbzaGXVrcUnCISIlQo3Jky9pnrkxjy87dXDRiGn98dx4/7dS6V8VNwSEiJUq3pscwYVAGv+1Yj5c//YbuQyYzbt7asMuKKwoOESlxKqYk8adzm/HuLelUrZDCTS/P4oYXc1izWeteFQcFh4iUWC3rVGHMrekMPrMJWV+tp/uQLJ7P1rpX0abgEJESLTkxgRszjmfCgAxaH1eFe99fwPnDs5m3WuteRYuCQ0RKheOqlefFa9rxWJ/WfLdpJz2fmMpfPljAtl15hz9ZCkXBISKlhpnRs2VNPh6UQe92x/HM1BV0HzKZiQu+D7u0UkXBISKlzlHlk/nr+Sfx1s0dqFQ2metfzOHGlzR4XlQUHCJSarWtW5UP+nfi7swmTF6ynm4Pa9OooqDgEJFSLTkxgZu7Hs/EgRm/bBp13rBsvlylwfMjpeAQkbjw86ZRT1zWmrU/7aTXsKn8+f35bNXgeaEpOEQkbpgZ57SoyaRBGVx+Sl2en/Y13YdMZvx8zTwvDAWHiMSdo8ol83/nNeetmztyVLlkbnxpFte9kMPqTRo8LwgFh4jErTbHHc37t3Xi92c1IXvpD3QfMplnpiwnb4+WbT8UBYeIxLXkxARu6HI8EwZ24ZT6VfnLhwvp+UQ2c1ZuCru0mKXgEBEhMnj+3G9PZvjlbfhh6y7OG57NvWPms0XLtv8XBYeISMDMOOuk3zDpjgyubF+XF6Z/Tbchk/noyzXa8zwfBYeIyH4ql03mz72a884t6VSrkMLNr8zmuhdyWLVxe9ilxQQFh4jIQbQKlm3/w9knMm3ZBroPyWJk1jJ2x/nguYJDROQQkhITuK5zAyYO6kJ6w2r8dewiej6Rzeffxu+e5woOEZECqH10eZ6+Mo0RV7Rl47ZcLngyfvc8V3CIiBSQmZHZ/Fgm3RHZ8/yVT7/h9Icn8+Hc+Bo8V3CIiBTSL3ue90vnmMop9Ht1Nlc/P5OVP8bH4LmCQ0TkCLWoXYV3b0nnj+c0ZeaKH+k+dDIjJpf+wXMFh4jIr5CUmMC1neozcVAGXRql8sBHizj38anM+ubHsEuLGgWHiEgRqFmlHCOvTGNk37Zs3rGbC5+czj1vzWXjttywSytyCg4RkSJ0RrNjmTQogxu6NODNWas4fchkRs9aVaoGzxUcIiJFrEJKEr8/60Q+uK0T9aqV53dvzuHSkTP46vstYZdWJBQcIiJRcuJvKjP6po48cMFJLF67hTMfncKD4xaxI3dP2KX9KgoOEZEoSkgwerc7jn/dkcF5rWvx5L+X0X3oZD5e+H3YpR0xBYeISDGoVjGFf1zcklE3tKdsciLXvpDDjS/l8F0J3HUwqsFhZplmttjMlprZPQc55hIzW2Bm883s1Xztfw/aFprZY2ZmQfulZjY3+N6D0axfRKSotW9QjbH9O3NX5glMXrKebkMm83TW8hI19yNqwWFmicAw4EygKdDHzJrud0wjYDCQ7u7NgAFBe0cgHWgBNAdOBjLMrBrwEHB6cPyxZnZ6tK5BRCQayiQlcEvXhkwcmEH7BtW4f+zCYO5HyVg4MZo9jnbAUndf7u65wCig137HXA8Mc/eNAO6+Lmh3oCxQBkgBkoHvgQbAV+6+PjhuEnBhFK9BRCRq6lQtz7NXRRZOjMz9mMbgt+eyaXtsz/2IZnDUAlbme70qaMuvMdDYzLLNbIaZZQK4+3TgE2BN8DXe3RcCS4ETzKyemSUB5wF1DvTDzewGM8sxs5z169cf6BARkdD9snDioAyu71yfN3JWcdrDsT33I+zB8SSgEdAV6AM8bWZVzKwhcCJQm0jYnGZmnYOeyc3A68AU4GvggM+1uftId09z97TU1NSoX4iIyK9RISWJ/zm76T5zP3rH6NyPaAbHavbtDdQO2vJbBYxx993uvgJYQiRIzgdmuPtWd98KfAR0AHD39939FHfvACwOzhERKRV+nvvxtwtOYlEw9+PvMTb3I5rBMRNoZGb1zawM0BsYs98x7xLpbWBm1YnculoOfEtkMDzJzJKBDGBhcFyN4L9HA7cAz0TxGkREil1CgtEnmPvRq1UthgdzP/61KDbmfkQtONw9D7gVGE/kL/033H2+md1nZj2Dw8YDG8xsAZExjTvdfQMwGlgGfAnMAea4+/vBOY8Gx2cDD7i7ehwiUipVq5jCw5f8Z+7HNc/HxtwPi9XBl6KUlpbmOTk5YZchInLEcvP28szU5Tz28VckmDGwW2N+m16P5MTo3Tgys1nunrZ/e9iD4yIiUgCxNPdDwSEiUoIceO7Hl8U690PBISJSwvz33I+VnP7wZN4qprkfCg4RkRIq/9yPutXKc0cw92PpuujO/VBwiIiUcMU990PBISJSCuSf+9Gz5X/mfixeW/S9DwWHiEgpkn/uR/3qFah9dLki/xlJRf6OIiISuvYNqtG+QbWovLd6HCIiUigKDhERKRQFh4iIFIqCQ0RECkXBISIihaLgEBGRQlFwiIhIoSg4RESkUOJiIyczWw98c4SnVwd+KMJySjp9Hv+hz2Jf+jz2VRo+j7runrp/Y1wEx69hZjkH2gErXunz+A99FvvS57Gv0vx56FaViIgUioJDREQKRcFxeCPDLiDG6PP4D30W+9Lnsa9S+3lojENERApFPQ4RESkUBYeIiBSKguMQzCzTzBab2VIzuyfsesJiZnXM7BMzW2Bm883s9rBrigVmlmhmn5vZB2HXEjYzq2Jmo81skZktNLMOYdcUFjMbGPyezDOz18ysbNg1FTUFx0GYWSIwDDgTaAr0MbOm4VYVmjzgDndvCrQH+sXxZ5Hf7cDCsIuIEY8C49y9CdCSOP1czKwW0B9Ic/fmQCLQO9yqip6C4+DaAUvdfbm75wKjgF4h1xQKd1/j7rODP28h8pdCrXCrCpeZ1QbOBp4Ju5awmdlRQBfgWQB3z3X3TaEWFa4koJyZJQHlge9CrqfIKTgOrhawMt/rVcT5X5YAZlYPaA18GnIpYXsEuAvYG3IdsaA+sB74Z3Dr7hkzqxB2UWFw99XAP4BvgTXAZnefEG5VRU/BIQVmZhWBt4AB7v5T2PWExczOAda5+6ywa4kRSUAb4El3bw1sA+JyTNDMjiZyZ6I+UBOoYGZXhFtV0VNwHNxqoE6+17WDtrhkZslEQuMVd3877HpClg70NLOvidzCPM3MXg63pFCtAla5+8+90NFEgiQedQNWuPt6d98NvA10DLmmIqfgOLiZQCMzq29mZYgMcI0JuaZQmJkRuX+90N2HhF1P2Nx9sLvXdvd6RP6/+Je7l7p/VRaUu68FVprZCUHT6cCCEEsK07dAezMrH/zenE4pfFAgKewCYpW755nZrcB4Ik9GPOfu80MuKyzpQF/gSzP7Imj7vbuPDa8kiTG3Aa8E/8haDlwdcj2hcPdPzWw0MJvI04ifUwqXHtGSIyIiUii6VSUiIoWi4BARkUJRcIiISKEoOEREpFAUHCIiUigKDpEYZ2ZdtQKvxBIFh4iIFIqCQ6SImNkVZvaZmX1hZk8F+3VsNbOhwf4MH5tZanBsKzObYWZzzeydYI0jzKyhmU0yszlmNtvMjg/evmK+/S5eCWYli4RCwSFSBMzsROBSIN3dWwF7gMuBCkCOuzcDJgN/Ck55Ebjb3VsAX+ZrfwUY5u4tiaxxtCZobw0MILI3TAMis/lFQqElR0SKxulAW2Bm0BkoB6wjsuz668ExLwNvB/tXVHH3yUH7C8CbZlYJqOXu7wC4+06A4P0+c/dVwesvgHrA1KhflcgBKDhEioYBL7j74H0azf6433FHusbPrnx/3oN+dyVEulUlUjQ+Bi4ysxoAZlbVzOoS+R27KDjmMmCqu28GNppZ56C9LzA52F1xlZmdF7xHipmVL86LECkI/atFpAi4+wIz+wMwwcwSgN1APyKbGrULvreOyDgIwFXAiCAY8q8m2xd4yszuC97j4mK8DJEC0eq4IlFkZlvdvWLYdYgUJd2qEhGRQlGPQ0RECkU9DhERKRQFh4iIFIqCQ0RECkXBISIihaLgEBGRQvl/nWkt8DbHhzoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "losses = sf.reveal(losses)\n",
    "\n",
    "plot_losses(losses)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a58ff0dc-e2ac-4dfe-b562-4f3e42059c02",
   "metadata": {},
   "source": [
    "Finally, let's validate the model with AUC."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9bdffb27-4a44-4ee6-ab00-30d1b906fb38",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "auc=0.939731411726171\n"
     ]
    }
   ],
   "source": [
    "auc = validate_model(sf.reveal(W_), sf.reveal(b_), X_test, y_test)\n",
    "print(f'auc={auc}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac49ff9c",
   "metadata": {},
   "source": [
    "You may find the model from SPU trainning program achieve the same AUC as JAX program.\n",
    "\n",
    "This is the end of lab."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('py3.8')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "66d1547304beaba725027c44e57cc46fc747862fe9496520910412a3187eb35f"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
